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A method and apparatus for setting a marker on an object and tracking the 

position of the object 

FIELD OF THE INVENTION 

5 The field of this invention relates to digital imaging devices. 

BACKGROUND OF THE INVENTION 

Digital imaging devices typically have many automatic settings. For example, 
many digital cameras automatically set the focus, the aperture, and the exposure time. 

10 Some digital cameras allow the user to override or assist the automatic settings. For 
example, some digital cameras allow the user to set the area in the image that will be 
used by the camera to determine the focus. One way this is done is by centering the 
area in the viewfinder and pressing the shutter button halfway down. Once the camera 
has focused, the user re-frames the shot and presses the shutter button the rest of the 

15 way down. Other cameras allow the user to mark an object to be used as the focus 
area, for example see US patent 5,187,585 "Image sensing apparatus with settable 
focus detection area" which is hereby included by reference. One way that is used to 
select an object is for the user to center the object in the viewfinder or display and 
then press a control. Once an object has been selected a marker is displayed that 

20 identifies the object. There are many styles used for markers, one example is square 
brackets that is shown surroimding the selected object. Some cameras even allow 
multiple objects to be selected and marked. In many cameras today the camera tracks 
the selected object even when the camera is panned to reframe the scene. 
Unfortunately, cameras today lose track of a selected object when the camera is 

25 panned far enough that the object is no longer in the field of view of the device. 
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Therefore there is a need for a digital imaging device that can track the 
position of selected objects even when the selected objects are not currently in the 
field of view of the camera. 



5 SUMMARY OF THE INVENTION 

A digital imaging system that allows the user to select and mark objects with 
markers is disclosed. The digital imaging device tracks the position of the marked 
objects even when the marked objects are not in the field of view of the digital 
imaging system. 

10 Other aspects and advantages of the present invention will become apparent 

from the following detailed description, taken in conjimction with the accompanying 
drawings, illustrating by way of example the principles of the invention. 



BRIEF DESCRIPTION OF THE DRAWINGS 

15 Figure 1 is a block diagram of a digital imaging system that could implement 

an example embodiment of the present invention. 

Figure 2 is a flow chart for marking and then tracking an object in the scene in 
accordance with an example embodiment of the present invention. 

Figure 3 is a view of a scene with two objects marked with markers in an 
20 example embodiment of the present invention. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

An electronic block diagram of a typical digital imaging device is shown in 
25 Figure 1. Digital cameras today typically contain a photo-sensor (102) for capturing 
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images; a display area (104) for displaying the captured images and controlling the 
digital camera; a storage area (116) for storing the captured images; memory (108), 
for temporary manipulation of the captured images and for running the firmware of 
the camera; a processor (110) for controlling the camera, and some type of controls 
5 (106). Some digital cameras also include a microphone (1 14) for capturing audio clips 
along with the digital images. Some digital cameras include a speaker (118) and a 
digital signal processor (DSP 1 12). The controls (106) on digital cameras may include 
physical controls like buttons, rocker switches, and a keyboard, and virtual controls 
shown in the display area. The digital images, video clips and audio clips captured by 

10 the digital camera may be stored in memory (108) or may be moved to the storage 
area (116). Today the memory and the storage area are typically different types of 
devices. The memory is typically fast volatile memory and the storage area is 
typically slower non- volatile memory. In the future, as the speed of non- volatile 
memory increases, all the memory may be of the non-volatile type. Digital imaging 

15 devices typically have an input/output (I/O) channel (122). This I/O channel may be, 
for example, a USB bus, a SCSI bus, an IR link. Fire Wire, or a parallel link. The I/O 
channel is used to connect the digital imaging device to other computer systems or 
networks. Some digital imaging devices connect to other computer systems using a 
camera dock. Digital cameras may also contain a wireless link (120) to the Internet, 

20 for example a cell phone. 

Some digital cameras have more than one display area, for example a camera 
may have an LCD display on the back of the camera and have a micro display used as 
a viewfinder. Both the LCD and the micro display can be used to display a real time 
view of the scene viewed by the camera. This also allows the camera to display 

25 additional information in the displays as the user frames the picture. One type of 
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information that may be displayed is a marker to show where, in the field of view, the 
camera is focusing. Some cameras allow the user to set or pick an object to use for 
focusing, for example see US patent 5,187,585 "Image sensing apparatus with settable 
focus detection area". In the 585 patent, an object can be designated, and the camera 
will use that object as the area to focus on, even if the object is moving. There are a 
number of well-known methods to designate an object. One way is to center the 
object in the viewfmder and then activate a control. Some cameras use the SI position 
of the shutter button as the control (Typically, the SI position is when the shutter 
button has been pressed half way down). Other cameras have controls that are 
different than the shutter button. 

Once an object has been designated, some cameras mark that object in the 
display area using a marker. There are many types of markers that can be used, for 
example square brackets can be shown surrounding the designated object. When the 
camera is moved to reframe the scene, the marker stays centered on the designated 
object. In this way, the designated object is tracked as its position shifts in the field of 
view. 

There are many ways to track objects in a scene. One way to track objects is 
disclosed in US patent 5,187,585 "Image sensing apparatus with settable focus 
detection area". In the 585 patent, an object detection module determines the image 
characteristics of designated objects. It then uses these image characteristics to track 
the position of the object, even when the object is in motion. Another way to track the 
position of objects is disclosed in the application "Pointing device for digital camera 
display'*, nxmiber 09/484667 filed Jan. 18, 2000, hereby incorporated by reference for 
all that it teaches. The 667 application teaches taking multiple exposures, or frames, 
of a scene, in quick secession. The device then compares changes between subsequent 
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images to calculate an amount and a direction of change for the motion of the device. 
Using the calculated motion of the device, the relative position of objects in the scene 
can be determined. For example, the device would use the first image or two to 
establish a fi^me of reference. By tracking the subsequent motion of the device 
compared to the initial frame of reference, the direction the device is currently 
pointing can be determined. Using the current pointing direction and the initial frame 
of reference, object positions can be tracked. With a known object position inside the 
frame of reference, the object can be marked with a marker whenever the object 
position is within the field of view of the device. 

In one example embodiment of the current invention, a scene would be viewed 
in an electronic display (200). An object in the scene would be designated and marked 
with a marker (202). A second view of the scene would be displayed, where the object 
is no longer visible (204). A third view of the scene would be displayed, where the 
object is visible (206). The object would be remarked with the marker in the third 
view (208). In one example embodiment, the device would track the position of the 
object as the field of view of the device changed. In another example embodiment, the 
device would track the position of the object as the object moved. In another example 
embodiment of the current invention, additional objects could be marked and tracked. 

Figure 3 shows an example embodiment for markers (302) marking two 
objects in a scene. 

In one example embodiment of the current invention, a digital device would 
image a view of a scene onto a photo-sensor (102). The digital device would display 
the view of the scene on a display (104). The digital device would contain at least one 
control (106) allowing user input into the digital device. The digital device would 
contain a processor (110) configured to monitor the control (106). The processor (110 
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or 1 12) would be configured to establish an initial frame of reference when detecting 
user input from the control (106). The initial frame of reference would be established 
by reading the view of the scene from the photo sensor (102). The processor (1 10 or 
112) would be configured to display markers in the display (104) at a predetermined 
location with respect to the initial frame of reference. For example, the markers could 
be located in the center of the field of view of the device when the initial frame of 
reference is established. The processor (1 10 or 1 12) would be configured to compare 
multiple sequential views of the scene, captured by the photo sensor (102), to track 
the movement of the device with respect to the initial frame of reference. Using this 
information the processor (1 10 or 1 12) would be configured to display the marker 
when the predetermined location is within the field of view of the device. 

In one example embodiment, a marker is positioned over a viewed (and 
captured) image (e.g., in the center of the display) and X and Y coordinates of this 
position are recorded. As subsequent images are captured during the tracking process, 
correlation computations are performed by the device to determine the change in 
direction and distance of the current captured image verses the previously captured 
image. Once the direction and distance of movement between the images are known, 
a new marker having different X and Y coordinates is determined. This new 
coordinate value is the new location of the original object in the current image. The 
object metrker is then moved to the new coordinate position. 

The above-described process continues as to each new image captured during 
tracking. The resulting effect is that the marker tracks the original object of the image 
in which it was positioned. Even if the marker, and therefore the object, "moves" off 
of the display, the object's X and Y coordinates can still be tracked in that new 
objects are identified and tracked in the same manner. In other words, relationships 
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(e.^., distance and direction) between tracked objects are known and therefore may be 
used such that a scene several screens-widths away from the ciurent viewed scene can 
still be tracked as the camera is panned. When an object that moved off the screen 
comes back into the live-view image area, its marker can then be re-enabled over the 
object. 

In one example embodiment of the current invention, a digital device would 
image a view of a scene onto a photo-sensor (102). The digital device would display 
the view of the scene on a display (104). The digital device would contain at least one 
control (106) allowing user input into the digital device. The digital device would 
contain a processor (110) configured to monitor the control (106). The processor (110 
or 1 12) would be configured to enable an object detection module when detecting 
user input from the control (106). The processor (1 10 or 1 12) would be configured to 
display markers in the display (104) indicating the object detected. The processor 
(1 10 or 1 12) would be configured to track the object across multiple views of the 
scene, captured by the photo sensor (102). 

The foregoing description of the present invention has been presented for 
purposes of illustration and description. It is not intended to be exhaustive or to limit 
the invention to the precise form disclosed, and other modifications and variations 
may be possible in light of the above teachings. This invention is not limited to 
marking object to help focus a digital device, this invention can be used to mark other 
usefril objects, for example but not limited to, light sources, areas of interest, and fiash 
points. The embodiment was chosen and described in order to best explain the 
principles of the invention and its practical application to thereby enable others skilled 
in the art to best utilize the invention in various embodiments and various 
modifications as are suited to the particular use contemplated. It is intended that the 
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appended claims be construed to include other alternative embodiments of the 
invention except insofar as limited by the prior art. 
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